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Summary 




This report examines the availability of college and career readiness data in American 
Samoa and the Commonwealth of the Northern Mariana Islands, describes the 
functionality of the student data systems based on Data Quality Campaign criteria, 
and presents profiles of recent high school graduating classes in each jurisdiction 
on the basis of available data. The study found that for high school graduates: 

In American Samoa students’ mean grade point average was 2.84, fewer than 
60 percent of students passed at least one semester of a higher level math 
course, and most students scored below basic proficiency in both math and 
reading on the Stanford Achievement Test 10th Edition (SAT-10). 

In the Commonwealth of the Northern Mariana Islands students’ mean grade 
point average was 2.81, 9.4 percent of students passed at least one quarter 
of Advanced Placement calculus, 38.7 percent passed an Advanced Placement 
English course, and most students scored at the average proficiency level in 
both math and reading on the SAT-10. 
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Summary 


Many jurisdictions use data about college and career readiness to help stakeholders under- 
stand whether students are on track to succeed in college and careers after high school 
graduation. For example, Hawaii includes the percentage of high school graduates from 
a particular school who later attend college in school-level feedback reports for principals 
and other stakeholders. 

In American Samoa and the Commonwealth of the Northern Mariana Islands, education 
stakeholders have identified high school graduates’ college and career readiness as a key 
concern. Although both jurisdictions are taking steps to improve their data systems, it 
is unclear what data are available that can be used to determine students’ college and 
career readiness. This study cataloged the availability of college and career readiness data 
in both jurisdictions, described the functionality of the student data systems based on Data 
Quality Campaign criteria (Data Quality Campaign, n.d.), and developed profiles of the 
2012/13 graduating high school class in American Samoa and of the 2013/14 graduating 
class in the Commonwealth of the Northern Mariana Islands. 

The study team identified a set of college and career readiness indicators used in Hawaii as a 
starting point to develop a profile of each jurisdiction’s graduating class (Hawaii P-20 Part- 
nerships for Education, 2015). The study team also identified additional indicators in the lit- 
erature on college and career readiness that are used by other states (Adelman, 2006; Hawley 
& Harris, 2005; Hein, Smerdon, & Sambolt, 2013; Mishkind, 2014; Porter & Polikoff, 2012). 

The study addressed college and career readiness separately in American Samoa and the 
Commonwealth of the Northern Mariana Islands. The study team identified the avail- 
able college and career readiness indicators in each jurisdiction and determined how many 
of the Data Quality Campaign’s 10 Essential Elements of Statewide Longitudinal Data 
Systems were in place. The study team also created a college and career readiness profile 
of each jurisdiction’s recent high school graduating class based on the identified indicators. 

In American Samoa: 

• Data were available to develop five college and career readiness indicators: cumula- 
tive grade point average, high school graduates, higher level math courses, higher 
level math course grades, and Stanford Achievement Test 10th Edition (SAT-10) 
math and reading proficiency. 

• The American Samoa Department of Education’s data system includes 6 of the 
Data Quality Campaign’s 10 essential elements: a unique student identifier; stu- 
dent-level enrollment, demographic, and program participation information; the 
ability to match individual students’ test records from year to year to measure aca- 
demic growth; a teacher identifier system; student-level transcript data, including 
information on courses passed and grades received; and a state data audit system 
that assesses data quality, validity, and reliability. 

• Among the 843 high school graduates in the sample, students’ mean grade point 
average was 2.84, fewer than 60 percent of students passed at least one semester of 
a higher level math course, and most students scored at the below basic proficiency 
level in math and reading on the SAT-10. 

• On average female students had a higher cumulative grade point average and 
higher SAT-10 reading proficiency than did male students, and a higher percentage 


of female students than of male students passed at least one semester of precalculus 
and calculus. 

In the Commonwealth of the Northern Mariana Islands: 

• Data were available to develop six college and career readiness indicators: 
Advanced Placement courses, cumulative grade point average, high school gradu- 
ates, higher level math courses, higher level math course grades, and SAT-10 math 
and reading proficiency. 

• No systemwide student longitudinal data system was in place. Separate offices and 
departments collect and manage student enrollment data and SAT-10 results, and 
individual high schools store academic information. 

• The Commonwealth of the Northern Mariana Islands’ data systems include 3 
of the Data Quality Campaign’s 10 essential elements: studentdevel enrollment, 
demographic, and program participation information; the ability to match indi- 
vidual students’ test records from year to year to measure academic growth; and 
student-level transcript data, including information on courses passed and grades 
received. 

• Among the 587 high school graduates in the sample, 9.4 percent of students passed 
at least one quarter of Advanced Placement calculus and 38.7 percent passed an 
Advanced Placement English course, students’ mean grade point average was 2.81, 
and most students scored at the average proficiency level on both the SAT-10 math 
and reading exam. 

• On average female students had higher cumulative grade point averages and higher 
SAT-10 reading proficiency than did male students, and a higher percentage of 
female students than of male students passed at least one quarter of precalculus 
and Advanced Placement English. 
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Why this study? 


Education stakeholders in the Regional Educational Laboratory (REL) Pacific Region and 
across the United States are concerned about the college and career readiness of high school 
graduates. Many states use college and career readiness indicators, such as student performance 
on standardized exams, to inform policy and practice. These indicators can help stakeholders 
determine whether students are on track to succeed in college and careers. For example, the 
college and career readiness indicators report for 2014 for Hawaii includes the percentage of 
high school graduates from particular schools who later attend college in schooUevel feedback 
reports for principals and other stakeholders (Hawaii P-20 Partnerships for Education, 2015). 

In American Samoa and the Commonwealth of the Northern Mariana Islands it remains 
unclear whether data are available to construct longitudinal indicators of college and 
career readiness. American Samoa recently established a longitudinal data system that 
may allow for the construction of longitudinal profiles of high school graduates. The Com- 
monwealth of the Northern Mariana Islands does not have a longitudinal data system, 
though it may be possible to construct longitudinal student records. 

This study furthers the goals of two groups of education stakeholders working with REL 
Pacific in those two jurisdictions: the American Samoa Alliance for College, Career, and Life 
Readiness and the Commonwealth of the Northern Mariana Islands Alliance for College 
and Career Readiness. These research alliances are composed of stakeholders from the 
K-12, college, government, and community sectors in each jurisdiction who have expressed 
concern about the number of high school graduates who appear unprepared to succeed in 
college and careers. In particular, alliance members have pointed to the high number of stu- 
dents placed into non-credit-bearing developmental courses when they start college, as well 
as the number of high school graduates who need remedial training in basic skills when they 
enter the workforce or the armed forces. These stakeholders also want to know whether male 
students and female students differ in their readiness for college and careers. 

Both jurisdictions have recently put in place policies and practices to improve the college 
and career readiness of high school graduates. Both jurisdictions have also adopted ver- 
sions of the Common Core State Standards that are intended to help improve college 
and career readiness (U.S. Department of Education, 2015). And both jurisdictions have 
recently adopted Achieve3000, a supplemental differentiated instruction program focused 
on improving nonfiction reading skills in order to boost college and career readiness by 
rapidly improving students’ reading proficiency (Achieve3000, 2015). 

The American Samoa Department of Education began implementing a five-year strate- 
gic plan in 2014/15. Its primary goal is to ensure that high school graduates are ready to 
succeed in college, careers, and life (American Samoa Department of Education, 2014). 
The plan calls for college and career readiness standards to be implemented across the 
K-12 curriculum. American Samoa recently was awarded additional funds to further 
develop its longitudinal data system by connecting the K-12 system to early childhood 
education and postsecondary data and to establish a transcript system for middle and high 
school students (Statewide Longitudinal Data Systems Grant Program, 2015). 

In the Commonwealth of the Northern Mariana Islands both the local community college 
(Northern Marianas College) and the K-12 public school system have adopted strategic 
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plans that prioritize improving college and career readiness and success. Both plans stress 
the need for better collaboration and alignment across the two education systems and for 
better use of data to inform efforts to improve college and career readiness and success 
(Commonwealth of the Northern Mariana Islands Public School System, 2015; Northern 
Marianas College, 2014). As part of the strategic planning, a collaborative action plan will 
be developed spanning the K-12 and college systems and will be submitted to the North- 
ern Mariana Islands House of Representatives in 2017. 

This report provides stakeholders with baseline information about the college and career 
readiness of a recent graduating high school class in each jurisdiction before changes to 
policy and practice were put in place. Stakeholders may choose to develop similar yearly 
profiles of college and career readiness in the future to help gauge the impact of their 
reform efforts. This report also provides detailed information about the availability of 
college and career readiness data in each jurisdiction. 

What the study examined 


This study addressed four research questions separately for American Samoa and for the 
Commonwealth of the Northern Mariana Islands: 

1. What college and career readiness indicators are available? 

2. How many of the Data Quality Campaign’s 10 Essential Elements of Statewide Longi- 
tudinal Data Systems are in place (Data Quality Campaign, n.d.)? 

3. What was the college and career readiness profile of a recent high school graduating 
class based on the indicators identified in research question 1? 

4. Were there differences in the college and career readiness profiles of male and female 
students based on the indicators identified in research question 1? 

To address research question 1, the study team first identified indicators of college and career 
readiness used in Hawaii to characterize high school students’ college and career readiness. 
(Hawaii’s P-20 Partnerships for Education publishes an annual report that describes the college 
and career readiness of the state’s students based on those indicators; Hawaii P-20 Partner- 
ships for Education, 2015.) The study team also identified other indicators in the literature 
on college and career readiness that are used by other states and that could be of interest to 
research alliance members (box 1). The study team used these indicators to examine available 
indicators in American Samoa and the Commonwealth of the Northern Mariana Islands. 

To address research question 2 the study team used the 10 essential elements identified by the 
Data Quality Campaign — a national nonprofit organization whose goal is to support educa- 
tors in using information to improve students’ educational outcomes — as being necessary to 
build and maintain a statewide longitudinal data system that generates high-quality student 
data that stakeholders can use to track students from preschool to college (box 2). Many of the 
elements, such as student-level enrollment data and student-level SAT, ACT, and Advanced 
Placement exam data, are also necessary to understand students’ college and career readiness. 
The 10 essential elements were used in the study to systematically examine the availability of 
college and career readiness data and the functionality of the data systems in each jurisdiction. 
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Box 1. College and career readiness indicators examined in this study 


Advanced Placement courses Percentage of students who passed an Advanced Placement course in math or 
English. Students were considered to have passed an Advanced Placement course if they received a passing grade 
in one or more semesters or quarters. 

Attendance. Average percentage of days absent. 

College enrollment rate. Percentage of students who enroll in college. 

College-level or developmental math or English course enrollment. Percentage of students who enroll either in 
college-level (credit-bearing) or in developmental (non-credit-bearing) math or English courses as their first course in 
each subject. 

Cumulative grade point average Average grades received in all courses taken in high school, typically reported on 
a 0-4 scale. 

Diploma or certificate type. Percentage of students who received each type of diploma or certificate awarded by a 
jurisdiction’s department of education. 

Dual credit participation. Percentage of students participating in dual credit programs, which provide students with 
the opportunity to enroll in and receive credit for college courses while students are still in high school. 

High school graduates. Number of students who received a diploma or certificate of completion. 

Higher level math courses. Percentage of students who passed a course beyond algebra II (for example, precalculus 
or calculus). Students were considered to have passed a course if they received a passing grade in one or more 
semesters or quarters. 

Higher level math course grades. Percentage of students receiving each letter grade in a higher level math course. 
Final course grades were calculated on the basis of grades for each semester or quarter completed. 

On-time graduation rate. Percentage of students who receive a diploma after four years of high school. This is calcu- 
lated as the number of on-time graduates in a year divided by the number of students entering grade 9 for the first 
time minus the number of students transferring out plus the number of students transferring in. 

SAT/ACT scores. Average student scores on the SAT or ACT, two of the most common college admissions 
assessments. 

SAT-10 math and reading proficiency. The Stanford Achievement Test, Tenth Edition (SAT-10), is a standardized 
norm-referenced assessment that compares a student’s performance with a national reference group. The SAT-10 
assessment differs from the SAT college admission exam. In American Samoa the SAT-10 is administered in grade 
12, and scores are reported using four proficiency levels: below basic, basic, proficient, and advanced. In the Com- 
monwealth of the Northern Mariana Islands the SAT-10 is administered in grade 11, and results are reported in 
national stanine rankings, with three proficiency levels: below average (stanines 1, 2, 3), average (stanines 4, 5, 6), 
and above average (stanines 7, 8, 9). 

State assessment scores. Average student scores on statewide exams in a variety of different subjects, including 
math, reading, science, and social studies. 

Source: Author's compilation and Hawai'i P-20 Partnerships for Education (2015). 


To address research question 3, the study team used the indicators identified in research 
question 1 to construct the college and career readiness profiles of the 2012/13 high school 
graduating class in American Samoa and the 2013/14 high school graduating class in the 
Commonwealth of the Northern Mariana Islands. To address research question 4, the 
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Box 2. Data Quality Campaign’s 10 essential elements of statewide longitudinal data systems 

The 10 essential elements are: 

1. A unique student identifier. A single, unduplicated number assigned to an individual student that remains with that 
student from kindergarten through high school and connects student data across key databases across years. 

2. Student-level enrollment, demographic, and program participation information. Information such as attendance, 
special education status, gifted and talented education status, career and technical education participation, 
and eligibility for the federal school lunch program. 

3. The ability to match individual students’ test records from year to year to measure academic growth and the 
ability to disaggregate the results by individual test item and objective. 

4. Information on untested students and the reasons why they were not tested. 

5. A teacher identifier system with the ability to match teachers to students by classroom and subject. 

6. Student-level transcript data, including information on courses completed and grades received from middle and 
high school. 

7. Student-level college readiness test scores such as scores on SAT, SAT II, ACT, Advanced Placement (AP), and 
International Baccalaureate (IB) exams. 

8. Student-level graduation and dropout data. 

9. The ability to match student records between the prekindergarten-to-grade 12 and postsecondary systems. 

10. A state data audit system assessing data quality, validity, and reliability 

Source: Data Quality Campaign website (http://dataqualitycampaign.org/why-education-data/state-progress/). 


study team explored whether male or female students in each jurisdiction were more likely 
to be prepared for college and careers. 

Appendix A provides a review of the literature on college and career readiness, with a 
focus on indicators of college and career readiness used in this study. Box 3 summarizes the 
data and methods used in the study, and appendixes B and C offer detailed information on 
the data and methods used for each jurisdiction. 


Box 3. Data, sample, and methods 
Data 

American Samoa. The American Samoa Department of Education’s Integrated Data Services Office provided stu- 
dent-level data on all 2012/13 high school graduates in American Samoa. Data included students’ demographic 
characteristics (gender, race/ethnicity, and home language) and information on enrollment status, coursework and 
grades, attendance, and scores on the Stanford Achievement Test 10th Edition (SAT-10). 

Commonwealth of the Northern Mariana Islands. The Northern Mariana Islands Public School System provided 
student-level data on 2013/14 high school graduates. Data included students’ demographic characteristics (gender, 
race/ethnicity, and home language), SAT-10 scores, and paper high school transcripts. 

Sample 

American Samoa. The sample included 843 graduates from the 2012/13 school year. American Samoa had 879 grad- 
uates in 2012/13, but data records for all four years of high school were not available for 36 graduates (4 percent), 
who were therefore excluded from the study. Students in the sample graduated from six public high schools. 

(continued) 
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Box 3. Data, sample, and methods (continued) 


Commonwealth of the Northern Mariana Islands. The sample included 587 graduates from the 2013/14 school 
year. Students in the sample graduated from three public high schools on the island of Saipan (Kagman High 
School, Marianas High School, and Saipan Southern High School). The sample did not include graduates from high 
schools on the outer islands of Rota and Tinian because those schools did not provide transcripts for analysis. 
Though the exact number of graduates for whom data were not provided is unavailable, approximately 5-10 percent 
of the population of the Commonwealth of the Northern Mariana Islands lives on those two outer islands. 

Methods for research questions 1 and 2 

To determine the availability of college and career readiness data essential elements in each jurisdiction, the study 
team interviewed stakeholders formally and informally, designed and negotiated data-sharing agreements, direct- 
ly examined electronic data systems, exchanged emails, reviewed publicly available education reports and other 
artifacts, and examined high school transcripts. The study team also reviewed student-level data provided by each 
jurisdiction, and interviewed stakeholders formally and informally. 

Methods for research questions 3 and 4 

On the basis of available college and career readiness indicators identified in research question 1, the study team 
generated descriptive statistics to construct college and career readiness profiles for the 2012/13 high school 
graduating class in American Samoa and the 2013/14 high school graduating class in the Commonwealth of the 
Northern Mariana Islands. 

For each indicator, descriptive statistics were generated for male and female students. Chi-square tests were 
performed to determine whether differences in the percentages of male and female students for categorical indi- 
cators were significant. Finally, independent sample t-tests were conducted to examine differences by gender for 
continuous indicators. 

For additional information about the American Samoa data, including demographic characteristics of the 
sample, see appendix B; see appendix C for more about the Commonwealth of the Northern Mariana Islands data. 


What the study found 


This section describes results for the four research questions first for American Samoa and 
then for the Commonwealth of the Northern Mariana Islands. 

Data to construct five college and career readiness indicators were available in American Samoa 

Data were available to construct five college and career readiness indicators: cumulative 
grade point average, high school graduates, higher level math courses, higher level math 
course grades, and SAT-10 math and reading proficiency (table 1). 

Data were unavailable for five indicators: college enrollment, college-level or developmen- 
tal math or English course enrollment, dual credit participation, on-time graduation (data 
on transfers and dropouts were not systematically tracked), and SAT/ACT scores. 

Several indicators were considered not applicable in American Samoa. The number of 
students taking Advanced Placement courses was not applicable because the American 
Samoa Department of Education does not offer these courses. While information on 
diploma type was available, American Samoa offers only one type of high school diploma, 
which limits the usefulness of this indicator. And American Samoa does not administer a 
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territorial assessment comparable to state assessments in the mainland United States, so 
state assessment scores were considered not applicable as well. 


Attendance data were available, but data managers reported concerns about data accura- 
cy (American Samoa Department of Education data manager, personal communication, 
2014). In some cases attendance data were entered into the electronic data systems by high 
school counselors, who often waited weeks or months to input the data. In other cases 
schools reported that 100 percent of students were in attendance for whole quarters of the 
school year. New data processes have since been put in place systemwide, and the quality of 
attendance data has been characterized as “improving” (American Samoa Department of 
Education data manager, personal communication, 2014). But the study team determined 
that the attendance data from 2012/13 were not of high enough quality to include in the 
college and career profiles developed for this study. 

The American Samoa Department of Education’s longitudinal data system includes 6 of the Data 
Quality Campaign’s 10 essential elements 

The American Samoa Department of Education administers and maintains student data 
using one centralized comprehensive electronic student data system under the auspices of 
the Integrated Data Services Office. The Integrated Data Services Office is responsible for 
collecting, managing, and reporting all K-12 student data (American Samoa Department 
of Education, 2014). The staff also maintains a data audit system to ensure that data in the 
system are high quality, valid, and reliable. 
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Table 1. Availability of college and career readiness indicators in American Samoa, 
2012/13 

1 College and career readiness indicator 

Available 1 

Advanced Placement courses 

No a 

Attendance 

No b 

College enrollment rate 

No 

College-level or developmental math or English course enrollment 

No 

Cumulative grade point average 

Yes 

Diploma or certificate type 

No c 

Dual credit participation 

No 

High school graduates 

Yes 

Higher level math courses 

Yes 

Higher level math course grades 

Yes 

On-time graduation rate 

No 

SAT/ACT scores 

No 

SAT-10 math and reading proficiency 

Yes 

State assessment scores 

No d 


a. No Advanced Placement courses are offered in American Samoa. 

b. Because of concerns about the quality of attendance data from 2012/13, the study team decided not to 
include attendance data as an indicator. 

c. Data were available, but because American Samoa offers only one type of diploma, the usefulness of this 
indicator would be limited. 

d. American Samoa does not administer a territorial assessment comparable to state assessments used in 
the mainland United States, so state assessment scores were considered not applicable. 

Source: Authors' analysis of 2012/13 data from the American Samoa Department of Education. 
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The data system uses a territorywide student identifier for each student, which allows reli- 
able recordkeeping across multiple school years (table 2). When students register for school 
in American Samoa, they are assigned a unique identification number, which is then used 
for all student records. The identification number allows the American Samoa Depart- 
ment of Education to link K-12 student information across all types of data, including 
enrollment, test data, teacher identifiers, and course completion (transcript). 


Information on untested students is not tracked. For example, if students were missing 
SAT-10 data, it would be unclear whether they were excused from testing, they were 
absent, or the data were lost. Nor does the system track student-level SAT or ACT data 
or data on graduation, dropouts, and transfers. Unless another school requests a student’s 
school records, there is no way of knowing that a student transferred to another school or 
dropped out of school (American Samoa Department of Education data manager, personal 
communication, 2014). P-12 and higher education data cannot be matched. 

Although the longitudinal data system contains data from 2007 to the present, data 
entered before 2012/13 are considered less trustworthy. However, according to representa- 
tives of the Integrated Data Services Office, extensive quality control measures are being 
developed and implemented for all new data entering the system. For example, since 2013 
a data steward has been in place whose primary responsibility is to understand the quality 
of data entering the system and to intervene to ensure that data are entered accurately 
and in a timely fashion. In some cases, permission to enter data into the system has been 
revoked until the quality of the data being entered has improved through training. The 
American Samoa Department of Education has also begun establishing data standards, 
offering regular training for school staff charged with entering data, and looking for other 
ways to improve its data auditing system. 
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Table 2. Availability of the Data Quality Campaign’s 10 essential elements of 
statewide longitudinal data systems in American Samoa, 2012/13 

1 Element 

Available 

1. A unique student identifier 

Yes 

2. Student-level enrollment, demographic, and program participation information 

Yes 

3. The ability to match individual students' test records from year to year to 
measure academic growth 

Yes 

4. Information on untested students 

No 

5. A teacher identifier system 

Yes 

6. Student-level transcript data, including information on courses completed and 
grades received 

Yes 

7. Student-level college readiness test scores 

No 

8. Student-level graduation and dropout data 

No 

9. The ability to match student records between the prekindergarten-to-grade 12 
and postsecondary systems 

No 

10. A state data audit system addressing data quality, validity, and reliability 

Yes 

Source: Authors' analysis based on essential elements from the Data Quality Campaign's website (http:// 
dataqualitycampaign.org/why-education-data/state-progress/) and 2012/13 data from the American Samoa 
Department of Education. 
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College and career readiness profiles of the 2012/13 high school graduating class in American 
Samoa 


This section presents college and career readiness profiles for the 2012/13 high school grad- 
uating class in American Samoa (tables 3-6). Only statistically significant gender differ- 
ences are discussed. 


The mean cumulative grade point average for students in American Samoa was 2.84. 
The percentage of students who received a cumulative grade point average in the range 
of 2.0-2.99 (47 percent) was higher than the percentage who received a cumulative grade 
point average in the range of 3.0-4.0 (42 percent; table 3). On average female students had 
a higher grade point average (3.05) than did male students (2.66). 

Fewer than 60 percent of students in American Samoa passed at least one semester of 
a higher level math course. Higher level math courses included precalculus and calculus. 1 
The criterion of one semester was used in order to understand the readiness of as many 
students as possible and because the literature is unclear whether the benefits of com- 
pleting higher level math courses require taking a full-year sequence. The percentage of 
students who passed at least one semester of precalculus (43 percent) was higher than the 
percentage of students who passed at least one semester of calculus (15 percent; table 4). A 
higher percentage of female students than of male students passed precalculus and calcu- 
lus. See appendix D for algebra II course completion rates and grades in American Samoa. 


In American 
Samoa, female 
students had a 
higher grade point 
average than did 
male students. A 
higher percentage 
of female students 
than of male 
students passed 
precalculus 
and calculus 


Table 3. Cumulative grade point average in American Samoa, by gender, 2012/13 

Gender 

Mean 

(standard 

deviation) 

0-0.99 

(percent) 

1.0-1.99 

(percent) 

2.0-2.99 

(percent) 

3.0-4.0 

(percent) 

All students (n = 841) 

2.84 (0.64) 

a 

a 

47.4 

42.0 

Female students (n = 385) 

3.05 (0.60)*** 

a 

a 

39.2 

56.1 

Male students (n = 456) 

2.66 (0.61)*** 

a 

a 

54.4 

30.0 


*** A t-test indicated a statistically significant relationship between cumulative grade point average and gen- 
der [f(839) = -9.32; p < .001], 

Note: Data on cumulative grade point average were unavailable for two students, 
a. Data have been suppressed to protect confidentiality of students. 

Source: Authors' analysis based on 2012/13 data from the American Samoa Department of Education. 





Table 4. Percentage of students who passed a higher level math course in 

American Samoa, by course and gender, 2012/13 

1 Gender 

Precalculus 

Calculus 

All students (n = 843) 

43.4 

14.7 

Female students (n = 386) 

58.3*** 

21.2*** 

Male students (n = 457) 

30.9*** 

g 2*** 


*** Chi-square tests indicated a statistically significant relationship between completing precalculus and 
gender [/(842, 843) 62.80; p < .001] and a statistically significant relationship between completing calculus 
and gender [ x 2 (842, 843) 16.60; p < .001], 

Source: Authors' analysis based on 2012/13 data from the American Samoa Department of Education. 
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Some 79 percent of students in American Samoa who enrolled in calculus received a 
grade ofB or higher. Among students who enrolled in precalculus, 31 percent received an 
A, 37 percent received a B, and 19 received a C (table 5). Among students who enrolled in 
calculus, 54 percent received an A, 26 percent received a B, and 12 percent received a C. 

Most students in American Samoa scored at the below basic proficiency level in math 
and reading on the SAT-10. Some 95 percent of graduates scored below proficient (that 
is, either basic or below basic) in math, and 96 percent scored below proficient in reading 
(table 6). A higher percentage of female students (6 percent) than of male students 
(3 percent) scored proficient or advanced in reading. 


Table 5. Precalculus and calculus grades in American Samoa, by gender, 2012/13 
(percentage of students) 

Course and gender 

A 

(90-100) 

B 

(80-89) 

C 

(70-79) 

D 

(60-69) 

F 

(0-59) 

Precalculus 

All students (n = 372) 

30.6 

36.6 

18.5 

11.0 

3.2 

Female students (n = 228) 

35.1 

34.6 

18.9 

8.8 

2.7 

Male students (n = 144) 

23.6 

39.6 

18.1 

14.6 

4.2 

Calculus 

All students (n = 125) 

53.6 

25.6 

12.0 

5.6 

3.2 

Female students (n = 83) 

53.0 

21.7 

14.5 

a 

a 

Male students ( n = 42) 

54.8 

33.3 

a 

a 

a 


Note: Percentages may not sum to 100 percent because of rounding. Chi-square tests indicated no statisti- 
cally significant relationship between precalculus grades and gender [/(371, 372) 5.353; p = .25] or between 
calculus grades and gender [j^ 2 (124, 125) 1.593; p = .66], 

a. Data have been suppressed to protect confidentiality of students. 

Source: Authors' analysis based on 2012/13 data from the American Samoa Department of Education. 


Table 6. SAT-10 proficiency in math and reading in American Samoa, by gender, 
2012/13 (percentage of students) 


Exam and gender 

Below basic 

Basic 

Proficient or 
advanced* 

Math 

All students (n = 808) 

74.8 

19.8 

5.4 

Female students (n = 372) 

74.2 

21.0 

4.8 

Male students (n = 436) 

75.2 

18.8 

6.0 

Reading 

All students (n = 812) 

73.3 

22.8 

3.9 

Female students (n = 374) 

69.0* 

25.4* 

5.6* 

Male students ( n = 438) 

76.9* 

20.5* 

2.6* 


* A chi-square test indicated a statistically significant relationship between SAT-10 reading proficiency and 
gender [ x 2 (811, 812) 10.55; p < .05], 

Note: A chi-square test indicated no statistically significant relationship between SAT-10 math proficiency 
and gender [/(807, 808) 0.97; p = .62)]. The number of students indicates the total number of students who 
received a score in math and reading on the SAT-10 in grade 12. Percentages may not sum to 100 percent 
because of rounding. 

a. Combined to protect confidentiality of students. 

Source: Authors' analysis based on 2012/13 data from the American Samoa Department of Education. 
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Data to construct six college and career readiness indicators were available in the Commonwealth 
of the Northern Mariana Islands 


Data were available to construct six college and career readiness indicators: Advanced Place- 
ment courses, cumulative grade point average, high school graduates, higher level math 
courses, higher level math course grades, and SAT-10 math and reading proficiency (table 7). 

Data for five indicators were unavailable: college enrollment, college-level or developmental 
math or English course enrollment, dual credit participation, on-time graduation rate (data 
on transfers and dropouts were not systematically tracked), and SAT/ACT scores. 


Some indicators were considered not applicable or unreliable. While information on 
diploma type was available, only one type of diploma is offered, which limits the useful- 
ness of the diploma or certificate type indicator. In addition, the Commonwealth of the 
Northern Mariana Islands does not administer a territorial assessment comparable to state 
assessments in the mainland United States, so state assessment scores were not applicable 
either. Attendance data were inconsistently reported and not available for many of the 
high school graduates in the study sample. 

Data systems in the Commonwealth of the Northern Mariana Islands include 3 of the Data Quality 
Campaign’s 10 essential elements 

The Northern Mariana Islands Public School System has no centralized student data 
system. Several offices collect, manage, and report on student data. The Records Manage- 
ment and Data Center Department manages enrollment and demographic data (such as 


In the 

Commonwealth 
of the Northern 
Mariana Islands, 
data were available 
to construct 
six college and 
career readiness 
indicators, 
unavailable for 
five indicators, 
and unreliable or 
not applicable for 
three indicators 


Table 7. Availability of college and career readiness indicators in the 

Commonwealth of the Northern Mariana Islands, 2013/14 

1 College and career readiness indicator 

Available j 

Advanced Placement courses 

Yes 

Attendance 

No a 

College enrollment rate 

No 

College-level or developmental math or English course enrollment 

No 

Cumulative grade point average 

Yes 

Diploma or certificate type 

No b 

Dual credit participation 

No 

High school graduates 

Yes 

Higher level math courses 

Yes 

Higher level math courses grades 

Yes 

On-time graduation rate 

No 

SAT/ACT scores 

No 

SAT-10 math and reading proficiency 

Yes 

State assessment scores 

No c 


a. Individual high schools collect attendance data, but the study team decided not to include attendance data 
as an indicator because discussions with principals suggested that the data were unreliable. 

b. Data were available, but because the Commonwealth of the Northern Mariana Islands offers only one type 
of diploma, the usefulness of this indicator would be limited. 

c. The Commonwealth of the Northern Mariana Islands does not administer a territorial assessment comparable 
to state assessments used in the mainland United States, so state assessment scores were not applicable. 

Source: Authors' analysis based of 2013/14 data from the Northern Mariana Islands Public School System. 
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Table 8. Availability of the Data Quality Campaign’s 10 essential elements in the 
Commonwealth of the Northern Mariana Islands, 2013/14 

1 Element 

Available | 

1. A unique student identifier 

No 

2. Student-level enrollment, demographic, and program participation information 

Yes 

3. The ability to measure individual students' test records from year to year to measure 
academic growth 

Yes 

4. Information on untested students 

No 

5. A teacher identifier system 

No 

6. Student-level transcript data, including information on courses completed and grades 

received 

Yes 

7. Student-level college readiness test scores 

No 

8. Student-level graduation and dropout data 

No 

9. The ability to match student records between the prekindergarten-to-grade 12 and 
postsecondary systems 

No 

10. A state data audit system assessing data quality, validity, and reliability 

No 

Source: Authors' analysis based on essential elements from the Data Quality Campaign's website (http:// 
dataqualitycampaign.org/why-education-data/state-progress/) and 2013/14 data from the Northern Mariana 
Islands Public School System. 


school, age, gender, grade level). The Office of Accountability, Research, and Evaluation 
stores SAT-10 results provided by Pearson. And each high school stores academic informa- 
tion (such as courses taken, grades received, grade point average, and attendance). In some 
cases, students did not have a unique identification number. Stakeholders reported that stu- 
dents sometimes received new identification numbers when they left school and registered 
at the same school or at a different school. The public school system is working to develop 
a centralized longitudinal data system with unique student identification numbers and has 
purchased a software module that allows the data to be more centralized across schools. 


Data were available on student-level enrollment, demographic characteristics, and program 
participation; students’ test records; and student-level transcript data, including informa- 
tion on courses completed and grades received (table 8). Teacher information was collect- 
ed but not systematically tracked across all courses and subjects, and a student-teacher 
match was not possible. Information on untested students and SAT, ACT, and Advanced 
Placement exam data were not available. Graduation and dropout data were considered 
unreliable according to high school principals (Northern Mariana Islands high school 
principals, personal communication, 2015). The Northern Mariana Islands Public School 
System does not have a data audit system in place. 


The Northern 
Mariana Islands 
Public School 
System has no 
centralized student 
data system 


College and career readiness profiles of the 2013/14 high school graduating class in the 
Commonwealth of the Northern Mariana Islands 


This section presents college and career readiness profiles for the 2013/14 high school grad- 
uating class in the Commonwealth of the Northern Mariana Islands (tables 9-13). 

Fewer than 10 percent of students in the Commonwealth of the Northern Mariana 
Islands passed at least one quarter of Advanced Placement calculus, and nearly 
40 percent completed at least one quarter of an Advanced Placement English course. 
Students were considered to have passed an Advanced Placement course if they passed 
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Table 9. Percentage of students who passed Advanced Placement calculus and 
English courses in the Commonwealth of the Northern Mariana Islands, by course 


and gender, 2013/14 

Gender 

Advanced Placement calculus 3 

Advanced Placement English j 

All students (n = 587) 

9.4 

38.7 

Female students (n = 279) 

10.8 

46.6*** 

Male students (n = 308) 

8.1 

31.5*** 


*** A chi-square test indicated a statistically significant relationship between Advanced Placement English 
course completion and gender [/(586, 587) 14.08; p < .001], 

Note: A chi-square test indicated no statistically significant relationship between Advanced Placement calculus 
course completion and gender [/(586, 587) 1.198; p = .27]. 

a. Only two high schools offered Advanced Placement calculus. Of the 446 graduates in these two schools in 
the study sample, 12.3 percent passed at least one quarter of an Advanced Placement calculus course. 

Source: Authors' analysis based on 2013/14 data from the Northern Mariana Islands Public School System. 


at least one quarter. 2 Some 9 percent of students passed at least one quarter of Advanced 
Placement calculus (table 9). 3 Some 39 percent of graduates passed an Advanced Place- 
ment English course (which was offered in all three high schools included in the study). 
A higher percentage of female students (47 percent) than of male students (32 percent) 
passed an Advanced Placement English course. 

The mean cumulative grade point average for students in the Commonwealth of the 
Northern Mariana Islands was 2.81. The percentage of students who received a cumula- 
tive grade point average of 3.0 or higher (47 percent) was higher than the percentage who 
received a cumulative grade point average in the range of 2.0-2.99 (33 percent). On average 
female students had a higher grade point average (3.10) than did male students (2.57). 

Fewer than 10 percent of students in the Commonwealth of the Northern Mariana Islands 
passed at least one quarter of a higher level math course. Some 5 percent of students passed 
precalculus, 4 and 9 percent passed calculus (table ll). 5 A higher percentage of female students 
(7 percent) than of male students (3 percent) passed precalculus. See appendix D for algebra II 
course completion and grades in the Commonwealth of the Northern Mariana Islands. 

Some 65 percent of students in the Commonwealth of the Northern Mariana Islands 
who enrolled in calculus received a B or higher. Among students who enrolled in pre- 
calculus, 36 percent received an A, and 40 percent received a B (table 12). Among the 


Table 10. Cumulative grade point average in the Commonwealth of the Northern 
Mariana Islands, by gender, 2013/14 


Gender 

Mean (standard 
deviation) 

0-0.99 

(percent) 

1.0-1.99 

(percent) 

2.0-2.99 

(percent) 

3.0-higher 

(percent) 

All students (n = 557) 

2.81 (0.87) 

2.0 

17.2 

33.4 

47.4 

Female students (n = 262) 

3.10 (0.76)*** 

a 

a 

34.4 

58.0 

Male students (n = 295) 

2.57 (0.90)*** 

2.4 

27.1 

32.5 

38.0 


*** A t-test indicated a statistically significant relationship between cumulative grade point average and gen- 
der [t(555) = 7.47; p < .001], 

Note: Data on cumulative grade point average were unavailable for 30 students, 
a. Data have been suppressed to protect confidentiality of students. 

Source: Authors’ analysis based on 2013/14 data from the Northern Mariana Islands Public School System. 


In the 

Commonwealth 
of the Northern 
Mariana Islands, 
female students 
had a higher grade 
point average 
than did male 
students, and a 
higher percentage 
of female students 
than of male 
students passed 
precalculus 
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Table 11. Percentage of students who passed higher level math courses in the 
Commonwealth of the Northern Mariana Islands, by course and gender, 2013/14 

1 Gender 

Precalculus 

Calculus i 

All students (n = 587) 

5.1 

9.4 

Female students (n = 279) 

7.2* 

10.8 

Male students (n = 308) 

3.2* 

8.1 


* A chi-square test indicated a statistically significant relationship between completing precalculus and gender 
]/(586, 587) 4.64; p < .05], 

Note: A chi-square test indicated no statistically significant relationship between completing calculus and 
gender [ x 2 (586, 587) 1.20; p = .27], 

Source: Authors' analysis based on 2013/14 data from the Northern Mariana Islands Public School System. 


Table 12. Percentage of students who received each grade in precalculus and 
calculus in the Commonwealth of the Northern Mariana Islands, 2013/14 


Course ABC 


Precalculus (n = 30) 36.7 40.0 a a 

Calculus (n = 55) 36.4 29.1 21.8 12.7 

Note: In two of the three schools included in the study the grading scale for honors and Advanced Placement 
courses differed from the grading scale for regular courses (see appendix C). 

a. Data have been suppressed to protect confidentiality of students. 

Source: Authors' analysis based on 2013/14 data from the Northern Mariana Islands Public School System. 


students who enrolled in calculus, 36 percent received an A, and 29 percent received a B. 
Because of the small number of students who enrolled in precalculus and calculus, data 
disaggregated by gender are not reported. 

Most students in the Commonwealth of the Northern Mariana Islands scored at the 
average proficiency level in math and reading on the SAT-10. Some 76 percent of students 


Table 13. Percentage of students who scored at each proficiency level in math and 
reading on the SAT-10 in the Commonwealth of the Northern Mariana Islands, by 
gender, 2013/14 


Exam and gender 

Below average 

Average 

Above average 

Math 

All students ( n = 460) 

24.1 

55.2 

20.7 

Female students (n = 223) 

25.6 

56.1 

18.4 

Male students (n = 237) 

22.8 

54.4 

22.8 

Reading 

All students (n = 457) 

21.0 

68.3 

10.7 

Female students (n = 221) 

16.3** 

69.2** 

14.5** 

Male students ( n = 236) 

25.4** 

67.4** 

7.2** 


** Chi-square tests indicated a statistically significant relationship between SAT-10 reading proficiency and 
gender [ x 2 (456,457) 10.23; p < .01]. 

Note: Chi-square test indicated no statistically significant relationship between SAT-10 math proficiency and gender 
[/(459.460) 1.50; p = .47]. The number of students indicates the total number of students who received a score 
in math and reading on the SAT-10 in grade 11. Percentages may not sum to 100 percent because of rounding. 

Source: Authors' analysis based on 2013/14 data from the Northern Mariana Islands Public School System. 
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scored average or above average in math on the SAT-10 math exam, and 79 percent scored 
average or above average in reading on the SAT-10 (table 13). A higher percentage of female 
students (15 percent) than of male students (7 percent) scored above average on reading. 

Implications of the study findings 


This study provides policymakers, practitioners, researchers, and other stakeholders with 
a comprehensive profile of the college and career readiness of recent high school gradu- 
ates in American Samoa and the Commonwealth of the Northern Mariana Islands. The 
results can serve as a baseline that both jurisdictions will be able to use to assess how 
well their changes to policy and practice are working to help more high school graduates 
succeed in college and careers. The study also brings up issues for stakeholders to consider. 

Stakeholders could consider Increasing the availability of college and career readiness indicators 

Although the data used to construct several college and career readiness indicators were 
available in both jurisdictions, data for other indicators were not. For example, on-time 
high school graduation rates, college enrollment rates, and the percentage of students who 
enroll in either college-level (credit-bearing) or developmental (non-credit-bearing) math 
or English courses as their first course in each subject are not available in either jurisdic- 
tion. Policymakers and other stakeholders need access to as much actionable information 
as possible to accurately determine the college and career readiness of high school gradu- 
ates in their jurisdictions. 


Although the data 
used to construct 
several college and 
career readiness 
indicators were 
available in both 
jurisdictions, 
data for other 
indicators were not 


Stakeholders could consider ways to increase the number of students who complete rigorous 
coursework 


In both jurisdictions the percentage of students who complete rigorous coursework in 
math and English is low, though the percentage was higher among female students than 
among male students. Both jurisdictions may consider changes to policy and practice that 
would increase the number of students who complete such coursework. Research indicates 
that rigorous coursework, which may include higher level math and Advanced Placement 
courses, may be associated with college and career readiness and success. 

Increasing the number of students who take higher level math courses is a strategy that 
many jurisdictions are adopting to improve college and career readiness (Buddin & Croft, 
2014). In both American Samoa and North Mariana Islands students are required to take 
four years of English but only three years of math; requiring a fourth year of math might 
increase the percentage of students who take higher level math courses such as precalculus 
or calculus. However, just raising the requirements for high school graduation, without 
providing students with more support in math courses, may not be effective in increasing 
college and career readiness (Buddin & Croft, 2014). 

Stakeholders could consider changes to the data systems to allow for longitudinal analyses of 
college and career readiness 

Both jurisdictions may want to use the results of this study to prioritize the data elements 
and the data system capacity they will add to their data systems in the next few years 
to support decisionmaking that may ensure that more students are ready for college and 
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careers. For example, developing the capacity to have one unique student identification 
number that stays with students across their high school years and into college and careers 
may be a high priority in both jurisdictions. With a single student identification number 
for each student at both the high school and college levels, stakeholders can use high 
school data to better predict readiness for college and career outcomes, including the rates 
of participation in developmental education, the number of new workers requiring remedi- 
al training in basic skills, and the satisfaction of workers in their career trajectories. 

Limitations of the study 


This study has five primary limitations. 

First, the sample did not include all 2012/13 public high school graduates in American 
Samoa. Students were included only if they had four years of public school system high 
school data available. Students who transferred into or out of the public school system 
during high school were thus excluded. It is unclear whether the college and career readiness 
of transfer students differs from that of remaining students. However, the sample included 
students who transferred within the public school system if four years of high school data 
were available. Even though external transfer students were excluded, the final sample still 
accounts for about 96 percent of 2012/13 public high school graduates in American Samoa. 

Second, the sample included data on students from only three of the five high schools in 
the Northern Mariana Islands Public School System. The three schools are on the island 
of Saipan, the most populated and largest island in the Mariana archipelago. The study 
team could not obtain data from two high schools located on outer islands, Rota High 
School and Tinian High School. About 90-95 percent of the population of the Com- 
monwealth of the Northern Mariana Islands resides on the island of Saipan. It is therefore 
likely that the number of graduates from high schools on Rota and Tinian is small. 

Third, both jurisdictions had missing data, including SAT-10 scores, high school cumula- 
tive grade point averages, and demographic data. For example, SAT-10 math scores were 
available for 808 of 843 (96 percent) graduates in American Samoa. It could not be deter- 
mined whether students for whom data were missing are more or less ready for college and 
career based on the indicators, compared with students for whom data were available. The 
amount of missing data was limited, however (see appendix B for details on missing data 
for American Samoa and appendix C for details on missing data for the Commonwealth of 
the Northern Mariana Islands). 

Fourth, the SAT-10 is no longer used in American Samoa and the Commonwealth of the 
Northern Mariana Islands because both jurisdictions have adopted new assessments. The 
American Samoa Department of Education created its own standards-based assessment 
and piloted it in 2014/15. The Northern Mariana Islands Public School System adopted 
the ACT Aspire in 2014/15, in part to provide more information about college and career 
readiness. Stakeholders in both jurisdictions may want to consider how scores on the 
SAT-10 measure similar or different aspects of college and career readiness compared with 
what the new assessments measure. 

Fifth, some interpretation by the study team was required in determining whether a Data 
Quality Campaign essential element was present. For example, the Northern Mariana 


Both jurisdictions 
had missing data, 
including SAT- 
10 scores, high 
school cumulative 
grade point 
averages, and 
demographic data 
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Islands Public School System was able to generate unique student identifiers (element 1), but 
the study team considered this element unavailable because a student who leaves school and 
re-enters later or transfers to a different school within the system is often given a new identi- 
fier. A consistent student identifier is not in place, even though the functionality is present. 
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Appendix A. Literature review on college and career readiness indicators 


Indicators identified for this study can be correlated with college and career readiness out' 
comes. For example, measures of high school academic achievement including grade point 
average and the completion of rigorous high school coursework can predict future enroll- 
ment and success in college. Dual credit participation is positively related to enrollment in 
college immediately after high school, as well as to an increased likelihood of remaining in 
college two years after high school, with significantly higher grade point averages and more 
college credits received, compared with nonparticipation (Karp, Calcagno, Hughes, Jeong, 
& Bailey, 2007). 

High school grade point average can be an indicator of college readiness, particularly when 
considered in conjunction with college aptitude test scores (Atkinson & Geiser, 2009; 
Porter & Polikoff, 2012). Porter and Polikoff (2012) found high school grade point average 
to be as good an indicator of college freshman grade point average as standardized tests 
designed to predict college success. Students’ gender has been linked to high school grade 
point average, with female students typically receiving higher grades than male students in 
all subjects (Voyer & Voyer, 2014). 

The completion of rigorous high school courses is also correlated with future college 
enrollment and success (Adelman, 2006; Atkinson & Geiser, 2009; Burton & Ramist, 
2001; Porter & Polikoff, 2012). College readiness has been linked to the successful com- 
pletion of specific courses, including Advanced Placement courses and higher level math 
courses, such as precalculus, calculus, trigonometry, or statistics. For example, Advanced 
Placement exam scores predict college grades and graduation rates (Ackerman, Kanfer, & 
Beier, 2013). Similarly, in a review of large datasets from high school graduating classes in 
1982 and 1992, Adelman (2006) found that completion of at least one higher level math 
course beyond algebra II — such as trigonometry, precalculus, or calculus — was associated 
with more than a two-time greater chance of students completing a bachelor’s degree. Suc- 
cessful completion of algebra II in high school has also been associated with assignment 
to credit-bearing or developmental math courses and overall college success (Jonas et al., 
2012; Adelman, 2006). 

Although more directly related to high school graduation rates, attendance also has been 
identified as an indicator of postsecondary success (Achieve, 2013; Cromwell, McClarty, & 
Larson, 2013; Hein, Smerdon, & Sambolt, 2013). Low attendance can be linked to lower 
high school graduation rates (Cromwell et al., 2013). Missing no more than 10 percent of 
school days per grade level is associated with on-track high school graduation (Allensworth 
& Easton, 2007). Also, low attendance rates could negatively affect coursework and grade 
point averages, which also are predictors of college readiness and success (Allensworth & 
Easton, 2007). 

In recent years researchers have used high school assessments to measure how well stu- 
dents meet college and career readiness standards (Achieve, 2011; Atkinson & Geiser, 
2009; Cohen, 2008; Porter & Polikoff, 2012; Wiley, Wyatt, & Camara, 2010). Some states 
have developed high school achievement tests to serve as indicators of college readiness 
(Atkinson & Geiser, 2009; Porter & Polikoff, 2012; Wiley et al, 2010). In an annual survey 
of career and college readiness, Achieve (2012) found that seven states use state-developed 
high school standardized tests capable of measuring students’ college and career readiness, 
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with another 11 states requiring national college admissions exams as part of their high 
school assessment systems. Gender differences on standardized tests scores are not as pro- 
nounced as differences in classroom grades. Research has shown that, on average, female 
students do better than male students in reading assessments, while male students outper- 
form female students in math (ACT, 2014). 

Some researchers consider college readiness and career readiness to be conceptually 
similar (ACT, 2006; Clancy et al., 2014; Hein et al., 2013), while others stress the differ- 
ences in the requirements for students to succeed in college or career settings (Achieve & 
National Association of State Directors of Career Technical Education Consortium, 2014; 
Conley, 2012; Conley & McGaughy, 2012). Indicators such as participation in Advanced 
Placement, performance on standardized assessments, dual credit, and on-time graduation 
rates may be relevant to both college and careers (Achieve, 2013; Achieve & National 
Association of State Directors of Career Technical Education Consortium, 2014; Glancy 
et al., 2014) while other indicators may better align with workplace readiness, such as par- 
ticipation in career and technical education courses, work-based learning, and employabil- 
ity assessments (Achieve & National Association of State Directors of Career Technical 
Education Consortium, 2014). 
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Appendix B. Data and methods for the American Samoa analysis 


This appendix describes the data and methods used to address the research questions for 
American Samoa and the demographic characteristics of the American Samoa sample. 

Research questions 1 and 2 

The study team conducted several interviews with district- and school-level administrators 
in American Samoa, including staff at the American Samoa Department of Education’s 
Integrated Data Services Office and the director of education. These interactions occurred 
in person, on the telephone, or by email. To determine whether an indicator or any of 
the Data Quality Campaign’s 10 essential elements were present, the study team reviewed 
notes from the interviews and examined the data systems. The study team’s initial findings 
were discussed with the American Samoa Alliance for College, Career, and Life Readiness 
and key stakeholders, such the director of the Integrated Data Services Office, to ensure 
their accuracy. The study team also asked the education leaders to identify individuals, 
such as data managers, who should be consulted to learn about data availability. The study 
team spoke with the data managers and their teams, guidance counselors, high school 
principals, and high school registrars in American Samoa. 

The study team did not use rating scales to judge the degree to which an indicator or 
an essential element was available, per guidance from the Data Quality Campaign (Data 
Quality Campaign, personal communication, 2015). 

Research questions 3 and 4 

This section presents information on data sources and the steps required to process the 
data from American Samoa. 

Data. The American Samoa Department of Education’s Integrated Data Services Office 
provided all data. Student-level data for the 2012/13 graduating class were provided in an 
Excel file, which included several spreadsheets with information on students’ demographic 
characteristics, coursework, grades, grade point averages, credits received, SAT-10 scores, 
and attendance. In the Excel file an academic year was split into two semesters. 

Data linking and merging. American Samoa uses unique student identification numbers, 
which the study team used to link all the spreadsheets into a comprehensive file. After 
data were linked, the study team reviewed and compared names, birthdates, and gender to 
ensure that the linking was done correctly and to identify duplicate cases. The study team 
consulted with data managers to understand why the data may have duplicate cases. Data 
managers reported that duplicate cases arise when students transfer out of the system and 
later return into the system. Working with the data managers, the study team removed 
duplicate cases with redundant information. In some cases, data across records with dupli- 
cate student identification numbers were merged to construct the most complete record for 
a student. 

Categorizing math courses. The study reported on enrollment in three higher level math 
courses: algebra II, precalculus, and calculus. Course names often varied by school, year, 
and semester. The study team reviewed all courses included in the data and created a list 
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of all courses that appeared to be math courses. Data managers at the Integrated Data Sen 
vices Office reviewed the list and clarified which courses should be identified as algebra II, 
precalculus, and calculus. 

Course grades. Students’ grades for all coursework were included in the electronic data 
files provided by the Integrated Data Services Office. 

Missing data Approximately 4 percent of graduates did not have four years of high school 
data. These students were excluded from the analysis. 

Demographic characteristics of the sample 

The 2012/13 high school graduates in American Samoa were predominately male 
(54 percent) and Samoan (97 percent) and speak Samoan at home (97 percent; table Bl). 


Table Bl. Demographic characteristics of 843 high school graduates in American 
Samoa, 2012/13 


Student characteristic 

Number 

Percent 

Gender 

Female students 

386 

45.8 

Male students 

457 

54.2 

Race/ethnicity 

Samoan 

816 

96.8 

Other 3 

27 

3.2 

Home language 

Samoan 

817 

96.9 

Other b 

26 

3.1 

a. Includes Fijian, Filipino, Other Asian, 

Other Pacific Islander, Tongan, and White. 



b. Includes English, Serbian, Tagalog, and Tongan. 

Source: Authors' analysis based on 2012/13 data from the American Samoa Department of Education. 
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Appendix C. Data and methodology for the 
Commonwealth of the Northern Mariana Islands analysis 


This appendix describes the data and methods used to address the research questions for 
the Commonwealth of the Northern Mariana Islands and the demographic characteristics 
of the Commonwealth of the Northern Mariana Islands sample. 

Research questions 1 and 2 

The study team conducted several interviews with district- and school-level administrators, 
including the data manager from the Northern Mariana Islands Public School System’s 
Records Management and Data Center and the data manager from the Office of Account- 
ability, Research, and Evaluation. These interactions occurred in person, on the telephone, 
or by email. 

To determine whether an indicator or any of the Data Quality Campaign’s 10 essential ele- 
ments of statewide longitudinal data systems were present, the study team reviewed notes 
from the interviews and examined the data systems. The study team’s initial findings were 
discussed with the Commonwealth of the Northern Mariana Islands Alliance for College 
and Career Readiness and key stakeholders, including the commissioner of education in 
the Commonwealth of the Northern Mariana Islands, to ensure their accuracy. The study 
team also asked the education leaders to identify individuals, such as data managers, who 
should be consulted to learn about data availability. The study team spoke with the data 
managers and their teams, guidance counselors, high school principals, and high school 
registrars in the Commonwealth of the Northern Mariana Islands. 

The study team did not use rating scales to judge the degree to which an indicator or 
an essential element was available, per guidance from the Data Quality Campaign (Data 
Quality Campaign, personal communication, 2015). 

Research questions 3 and 4 

Data sources. The Northern Mariana Islands Public School System provided three data 
sources for the study. The first source was paper high school transcripts, which includ- 
ed four years of coursework data, grades, grade point averages, attendance, and credits 
received for the 2013/14 graduating class. Transcripts were provided by school principals 
from three of the jurisdiction’s five public high schools. Two high schools from the outer 
islands of Tinian and Rota did not provide transcripts. The study team had the transcripts 
processed into Excel files. The second data source was the Records Management and Data 
Center Department, the office in charge of student data records. The office transmitted 
to the study team Excel files that contained demographic information — gender, birthdate, 
race/ethnicity, and home language — for all students enrolled in 2013/14 (those who grad- 
uated and those who did not). The third data source was an Excel file of SAT-10 results for 
grade 11 students in the 2012/13 school year from the Office of Accountability, Research, 
and Evaluation. 

Data linking and merging. Data linking and merging required additional steps for the 
Commonwealth of the Northern Mariana Islands student data, because no centralized 
data system exists and student identification numbers are used inconsistently. 
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Data from the three sources were merged into a final dataset. First, all paper transcripts 
had to be converted into Excel format and assigned a unique identification number gener- 
ated by the study team. Next, the study team used LinkPlus, which created a probability of 
likely matches based on students’ last and first names (the only common variables across 
all three databases), to link the transcript data, the demographic data, and the SAT-10 
data. 6 

The study team learned that duplicate cases were also a result of students who transferred 
in and out of schools. For example, two students transferred from Saipan Southern Fligh 
School to Marianas Fligh School in grade 12. When school principals sent the transcripts 
to the study team for the 2013/14 graduating class, these students’ transcripts appeared in 
both high school files. The study team cross-checked the list of potential transfer students 
with the high school principals to verify that students had transferred and were the same 
individual. 

Categorizing math courses. The study reported on enrollment in three math courses: 
algebra II, precalculus, and calculus. Course names often varied by school, year, and 
semester. The study team gave each high school principal in the Commonwealth of the 
Northern Mariana Islands a list of course names that appeared to be math courses. Each 
principal reviewed the list and identified which courses should be categorized as algebra II, 
precalculus, and calculus. 

Math courses offered at the three high schools. The availability of precalculus and 
Advanced Placement calculus courses varied by high school. Kagman High School offered 
only precalculus. Southern Saipan High School offered both precalculus and Advanced 
Placement calculus as higher level elective math courses. Marianas High School offered 
only Advanced Placement calculus as a higher level elective math course. 

Qrading scales. Different grading scales were used for math courses within and between 
schools. At Kagman High School, algebra II honors and pre-calculus courses were graded 
using a scale of A = 101-110, B = 91-100, C = 81-90, D = 71-80, and F = 0-70, while 
regular algebra II courses were graded using a scale of A = 93-100, B = 83-92, C = 73-82, 
D = 63-72, and F = 0-62. At Saipan Southern High School advanced algebra II and 
pre-calculus were graded using a scale of A = 93-100, B = 83-92, C = 73-82, D = 63-72, 
and F = 0-62, while Advanced Placement calculus was graded using a scale of A = 101— 
110, B = 91-100, C = 81-90, D = 71-80, and F = 0-70. At Marianas High School a single 
grading scale of A = 93-110, B = 83-92, C = 73-82, D = 63-72, and F = 0-62 was used 
regardless of whether the course was classified as Advanced Placement, honors, or regular 
algebra II. The study team used the appropriate school-level grade scale to ensure that 
the grades students received were consistent with how the school determined students’ 
academic competencies. 

Course grades. On the transcripts an academic year was split into four terms. A numeric 
grade was reported at the end of every term. Because the transcripts did not provide a final 
grade for each course, which could span multiple terms, final grades for algebra II, precalculus, 
and calculus were calculated by the study team. To calculate the final algebra II grade, the 
study team calculated the mean numeric grade for four terms of algebra II. That numeric 
grade was translated into a letter grade using the appropriate scales for that school and the 
level of the algebra II course (such as honors or regular). To calculate the final precalculus 
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and calculus grades, the study team calculated the mean numeric grade for each term com- 
pleted. It was not necessary to complete four terms of these elective courses to get a grade. If 
a student took only one term of precalculus or calculus, the grade reported in the study would 
be the letter grade equivalent of that numeric grade. If a student took two terms of pre- 
calculus or calculus, the mean of those two numeric grades was converted into a letter grade. 

Missing data. A total of 626 students began grade 12 in 2013/14 at the three schools 
examined in this study. Of these students, 39 students withdrew or transferred to other 
schools in grade 12 or did not meet the graduation requirements and had to repeat grade 
12. The study team did not receive transcripts for these 39 students. All transcripts provid- 
ed by the high schools included four years of data. Coursework information from a prior 
school was available on the transcripts even if a student had transferred from a school 
outside of the Northern Mariana Islands Public School System. 

Five percent of students did not have a cumulative grade point average on their transcripts. 
Approximately 22 percent of students were missing SAT-10 data. Also, 2 percent of stu- 
dents were missing data on their home language. 

Demographic characteristics of the sample 

The high schools in the Commonwealth of the Northern Mariana Islands serve a variety 
of students, including large numbers of Filipino, Chamorro, and Carolinian students but 
also students from Asia and other Pacific Islands (table Cl). The graduating class in the 
Commonwealth of the Northern Mariana Islands was predominantly male (53 percent) 
and spoke English at home (91 percent). 


Table Cl. Demographic characteristics of 587 high school graduates in the 
Commonwealth of the Northern Mariana Islands, 2013/14 


Student characteristic 

Number 

Percent 

Gender 

Female students 

279 

47.5 

Male students 

308 

52.5 

Race/ethnicity 

Asian 3 

46 

7.8 

Carolinian 

57 

9.7 

Chamorro 

183 

31.2 

Filipino 

228 

38.8 

Other Pacific lslander b 

58 

9.9 

Other 3 

15 

2.6 

Home language 

English 

534 

91.0 

Other 3 

41 

7.0 

Missing 

12 

2.0 

a. Includes Chinese, Japanese, and Korean. 


b. Includes Chuukese, Kosraean, Palauan, Pohnpeian, Samoan, and Yapese. 

c. Includes White and all other races/ethnicities not listed. 

d. Includes Chinese, Chuukese, Korean, Thai, and all other languages not listed. 

Source: Authors' analysis based on 2013/14 data from the Northern Mariana Islands Public School System. 
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Appendix D. Algebra II course completion and grades 


Algebra II course completion and grades were of interest to members of the American 
Samoa Alliance for College, Career, and Life Readiness and the Commonwealth of the 
Northern Mariana Islands Alliance for College and Career Readiness. In addition, research 
has shown that successful completion of algebra II is associated with college readiness and 
success (Adelman, 2006; Jonas et al, 2012). 

American Samoa 

In American Samoa 73 percent of students completed algebra II (table Dl). A higher 
percentage of female students (82 percent) than of male students (65 percent) completed 
algebra II. 

Among students who enrolled in algebra II, 23 percent received an A, 29 percent received 
a B, and 25 percent received a C (table D2). Female students received higher grades on 
average in algebra II than did male students, and a higher percentage of female students 
(60 percent) than of male students (43 percent) received an A or B. 

Commonwealth of the Northern Mariana Islands 

In the Commonwealth of the Northern Mariana Islands almost all students completed 
algebra II (99.5 percent), which is required for high school graduation (table D3). Three 
students did not take algebra II (two students enrolled directly into precalculus and cab 
cuius; the other student was in special education and was not required to take algebra II). 


Table Dl. Algebra II course completion in American Samoa, by gender, 2012/13 

1 Gender 

Percent of students who completed algebra II 1 

All students (n = 843) 

72.8 

Female students (n = 386) 

81.9*** 

Male students (n = 457) 

65.2*** 


*** A chi-square test indicated a statistically significant relationship between completing algebra II and gen- 
der [x 2 (842, 843) 29.35; p < .001], 

Note: Completion was defined as passing two semesters. 

Source: Authors' analysis based on 2012/13 data from the American Samoa Department of Education. 


Table D2. Algebra II grades in American Samoa, by gender, 2012/13 (percentage 
of students) 



A 

B 

c 

D 

F 

Gender 

(90-100) 

(80-89) 

(70-79) 

(60-69) 

(0-59) 

All students (n = 678) 

22.7 

28.8 

24.9 

14.3 

9.3 

Female students (n = 335) 

29.6** 

30.4** 

22.1** 

11.9** 

6.0** 

Male students (n = 343) 

16.0** 

27.1** 

27.7** 

16.6** 

12.5** 


** A chi-square test indicated a statistically significant relationship between algebra II and gender [/(677, 
678) 18.503; p < .01], 

Note: Forty-three students completed only one semester of algebra II (1 received a C, 1 received a D, and 41 
received an F). Percentages may not sum to 100 percent due to rounding. 

Source: Authors' analysis based on 2012/13 data from the American Samoa Department of Education. 
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Table D3. Algebra II course completion in the Commonwealth of the Northern 
Mariana Islands, by gender, 2013/14 


Gender Percent of students who completed algebra II 


All students (n = 587) 99.5 

Female students (n = 279) 100.0 

Male students (n = 308) 99.0 

Note: Chi-square test indicated no statistically significant relationship between completing algebra II and 
gender [/(586, 587) 2.731; p = .98], 

Source: Authors' analysis based on 2013/14 data from the Northern Mariana Islands Public School System. 


Among students who enrolled in algebra II, 15 percent received an A, 27 percent received 
a B, 32 percent received a C, and 26 percent received a D (table D4). No Fs were given in 
algebra II; students who failed the course retook the course until they received a passing 
grade, since completing the course was required for high school graduation. A higher per- 
centage of female students (53 percent) than of male students (33 percent) received an A 
or B in algebra II. 


Table D4. Algebra II grades in the Commonwealth of the Northern Mariana Islands, 
by gender, 2013/14 (percentage of students) 


Gender ABC 


All students (n = 584) 15.4 26.9 31.8 25.9 

Female students ( n = 279) 18.6*** 34.1*** 31.2*** 16.1*** 

Male students (n = 305) 12.5*** 20.3*** 32.5*** 34.8*** 


*** A chi-square test indicated a statistically significant relationship between completing algebra II and gen- 
der [x 2 (583, 584) 36.186; p < .001], 

Note: Completion was defined as passing all four quarters. Percentages may not sum to 100 percent due to 
rounding. 

Source: Authors' analysis based on 2013/14 data from the Northern Mariana Islands Public School System. 
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Notes 


1. American Samoa uses a semester system. For elective courses, such as precalculus and 
calculus, students could enroll in one or two semesters of the course. 

2. The Commonwealth of the Northern Mariana Islands uses a quarter system. For elec- 
tive courses, such as Advanced Placement calculus (the only Advanced Placement 
math course offered) and precalculus and calculus, students could enroll in one to four 
quarters of the course. 

3. The availability of calculus courses varied by high school. Only two of the three high 
schools included in the study — Marianas Fligh School, which accounted for approx- 
imately half the students in the sample, and Saipan Southern Fligh School, which 
accounted for 26 percent of students in the sample — offered calculus, but the values 
reported refer to the percentage of all students in the sample. 

4. The availability of precalculus courses varied by high school. Only two of the three 
high schools included in the study — Kagman High School, which accounted for 
24 percent of students in the sample, and Saipan Southern High School, which 
accounted for 26 percent of students in the sample — offered precalculus, but the values 
reported refer to the percentage of all students in the sample. 

5. The only calculus course offered in the Commonwealth of the Northern Mariana 
Islands Public School System in 2013/14 was Advanced Placement calculus. Only 
Southern Saipan High School and Marianas High School offered calculus. 

6. LinkPlus is an open source software program developed by the Centers for Disease 
Control and Prevention that allows for accurate linking across disparate data sources 
(Centers for Disease Control and Prevention, 2015). 
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Making Connections 

Studies of correlational relationships 


Making an Impact 

Studies of cause and effect 


What’s Happening 

Descriptions of policies, programs, implementation status, or data trends 


What’s Known 

Summaries of previous research 


Stated Briefly 

Summaries of research findings for specific audiences 


Applied Research Methods 

Research methods for educational settings 


Tools 

Help for planning, gathering, analyzing, or reporting data or research 



